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Abstract 



We introduce a notion of fc-convexity and explore polygons in the plane that 
have this property. Polygons which are /c-convex can be triangulated with fast yet 
simple algorithms. However, recognizing them in general is a 3SUM-hard problem. 
^N , We give a characterization of 2-convex polygons, a particularly interesting class, 

and show how to recognize them in 0(n log n) time. A description of their shape is 
\^ • given as well, which leads to Erdos-Szekeres type results regarding subconfigurations 

^^ . of their vertex sets. Finally, we introduce the concept of generalized geometric 

c/3 I permutations, and show that their number can be exponential in the number of 

2-convex objects considered. 



f^ ■ 1 Introduction 

o 

2^ I The notion of convexity is central in geometry. As such, it has been generalized in 

many ways and for different reasons. In this paper we consider a simple and intuitive 

(^ ■ generalization of convexity, which to the best of our knowledge has not been worked on. 

O . It leads to an appealing class of polygons in the plane with interesting structural and 

algorithmic properties. 



A set in Mf^ is convex if its intersection with every straight line is connected. This 
definition may be relaxed to directional convexity or D- convexity [1^123] . by considering 
only lines parallel to one out of a (possibly infinite) set D of vectors. A special case is 



*Partially supported by the FWF Joint Research Program 'Industrial Geometry' S9205-N12, Projects 
MEG MTM2006-01267, DURSI 2005SGR00692, Project Gen. Gat. DGR 2009SGR1040, DGR 2009SGR- 
1040, MEG MTM2009-07242, MEG MTM2008-04699-G03-02, and the bilateral Spain-Austria program 
'Acciones Integradas' ES 01/2008 and HU2007-0017. 

institute for Software Technology, University of Technology, Graz, Austria, oaichOist .tugraz.at 

■^Institute for Theoretical Gomputer Science, University of Technology, Graz, Austria, 
aurenOigi . tugraz . at 

§ Gomputer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 
Gambridge, USA, edemaineOmit . edu 

^Departament de Matematica Aplicada II, Universitat Politecnica de Gatalunya, Barcelona, Spain, 
Ferran . HurtadoOupc . edu 

llDepartamcnto de Matematicas, Univcrsidad de Alcala, Madrid, Spain, pedro.rajnos@uah.es 
**Instituto de Matematicas, Universidad Nacional Autonoma de Mexico, urrutiaOmatem.unam.mx 

1 



■■t.. 



Figure 1: Intersecting two 2-convex sets 

ortho- convexity j2^, where only horizontal and vertical lines are allowed. For any fixed D, 
the family of D-convex sets is closed under intersection, and thus can be treated in a sys- 
tematic way using the notion of semi-convex spaces [22], which is sometimes appropriate 
for investigating visibility issues. The D-convex hull of a set M is the intersection of 
all D-convex sets that contain M. If D is a finite set, this definition of a convex hull 
may lead to an undesirably sparse structure — an effect which can be remedied by using 
a stronger, functional (rather than set-theoretic) concept of D-convexity [23] . 

fc-Convex Sets We consider a different generalization of convexity. A set M in M*^ 
is called k- convex (with respect to transversal lines) if there exists no straight line that 
intersects M in more than k connected components. Throughout the paper we will use 
the term k- convex, for shorto. Note that 1-convexity refers to convexity in its standard 
meaning. To reformulate in terms of visibility, call two points x,y & M k-visible iixyHM 
consists of at most k components. Thus a set is A;-convex if and only if any two of its 
points are mutually /c-visible. Applications of this concept may arise from placement 
problems for modems that have the capacity of transmitting through a fixed number of 
walls [I1[I3]. 

Unlike directional convexity, /c-convexity fails to show the intersection property: The 
intersection of /c-convex sets is not /c-convex in general (for fixed k). Figure [1] gives an 
example. For /c > 2, a /c-convex set M may be disconnected, or if connected, its boundary 
may be disconnected. In this paper, we will restrict attention (with an exception in 
Section H]) to simply connected sets in two dimensions, namely, simple polygons in the 
plane. 

There are two notions of planar convexity that appear to be close to ours. One is 
k-point convexity [6|[32] which requires that for any k points in a set M in M^, at least one 
of the line segments they span is contained in M. Thus 2-point convex sets are precisely 
the convex sets. The other is k-link convexity [22], being fulfilled for a given polygon P 
if, for any two points in P, the geodesic path connecting them inside P consists of at 
most k edges. The 1-link convex polygons are just the convex polygons. While there is a 



^We face notational ambiguity. The term 'fc-convex' has, m.aybc not surprisingly, been used in different 
settings, namely, for functions [26], for graphs [5], and for discrete point sets [201133] . Also, the concept 
of fc-point convexity [32] has later been called fc-convexity in [B] . 



relation between fc-convexity and the former concept (as we will show in Section [2j), the 
latter concept is totally unrelated. 

We will study basic properties of k-convex polygons, in comparison to existing polygon 
classes and convexity concepts in Section |2j This offers an alternative to the approach 
in |1] to define 'realistic' polygons as those being guardable (visible) by at most k guards. 
We prove that given a simple polygon P, the problem of finding the smallest k such that 
P is k-convex (equivalently, to find the stabbing number of P) is 3SUM-hard. On the 
other hand, a recognition algorithm that runs in Oln^) time for a polygon with n vertices 
is easy to obtain. Interestingly, /c-convex polygons can be triangulated, by a quite simple 
method, in 0{n\ogk) time. An 0{nk) time complexity is achieved in [1] for /c-guardable 
polygons. 

The first nontrivial value, k = 2, deserves particular attention. Already in this case, 
a novel class of polygons is obtained. A characterization of 2-convex polygons is given in 
Section |3l It leads to an O(nlogn) time algorithm for recognizing such polygons. Note 
that 2-convex polygons add to the list of special classes of polygons [HE] that allow 
for simple 0{n) time triangulation methods. We also provide a qualitative description 
of their shape, which implies an Erdos-Szekeres type result, namely, that every 2-convex 
polygon with n vertices contains a subset of at least ^/n vertices in convex position, and 
that its vertex set can be decomposed into at most 2y/2n subsets in convex position. 



In Section m we turn our attention to general /c-convex domains. We give observations 
on the union and intersection properties of such domains, and elaborate on an attempt 
to generalize the notion of geometric permutations from convex sets to fc-convex sets. 
In contrast to the 0{m) bound in [10] on the number of geometric permutations of m 
convex sets, it turns out that the number of generalized geometric permutations can be 
exponential in m, already for 2-convex sets. For 2-convex polygonal domains, the number 
of generalized geometric permutations is Oln^), if n denotes the total number of their 
vertices. 

Various open questions are raised by the proposed concept of fc-convexity. We list 
those which seem most interesting to us, along with a brief discussion of our results, in 
Section O 

2 /c- Convex Polygons 

2.1 Basic properties 

We start with exploring some basic properties of A;-convex polygons, and compare them 
to existing polygon classes and related concepts. All geometric objects we will consider 
are closed sets in the Euclidean plane. 

Let P be a simple polygon, and denote by n the number of vertices of P. Two line 
segments e and e' are said to cross if e fl e' is a point in the relative interior of both e 
and e'. Clearly, a polygon P is /c-convex if every line segment with endpoints in P crosses 
at most 2(/c — 1) edges of P. The stabbing number [Tll|35] of a set of (interior-disjoint) 
line segments is the largest possible number of intersections between the set and a straight 
line. A polygon is /c-convex if and only if its stabbing number is at most 2/c. Thus, all 



our observations on /c-convexity could be reformulated in terms of stabbing numbers. 
Moreover, we have the following property, which we state for future reference. 

Lemma 1. If P is a k-convex polygon, then any polygon living on a subset of vertices of 
P is also k-convex. 





(a) 



(b) 



Figure 2: Quadratic 2- kernel construction 

The kernel of a simple polygon P is the set of points that see all the polygon. Its 
generalization to /c-convexity shows that 2-convexity is already significantly more complex 
than standard convexity. The k-kernel of P, denoted as Mk{P), is the set of points from 
which the entire polygon P is fc- visible. Note that P is fc-convex if and only if P = M^. 
While Ml is known to be a convex set which is computable in 0{n) time [21], M2 may 
have r2(n^) complexity: If we consider the 'spike' in Figure [2]^a), the wedge between 
the lines pq and pr is not part of M2. If we arrange such spikes along the boundary of 
a rectangle, as in Figure [2](b), we get a quadratic number of disconnected areas which 
are part of the 2-kernel. Therefore, any algorithm computing M2, or trying to check 
2-convexity via the comparison of P and M2 would have f2(n^) complexity in the worst 
case. 




Figure 3: Star-shaped versus 2-convex 



There is also no immediate relation to star-shaped polygons, i.e., polygons P with Mi 7^ 
Figure E] shows a polygon on the left hand side which is star-shaped but only |-convex. 




Figure 4: Guarding a 3-convex polygon 



On the right hand side, we see a polygon which is 2-convex but not star-shaped. Visually, 
2-convexity seems to be closer to convexity than is star-shapedness. Note that cutting a 
2-convex polygon with any straight line leaves (at most) three parts, each being 2-convex 
itself. This is not true in general for star-shaped polygons. 



While 2-convexity obviously restricts the winding number of a polygon [M], its link 
distance [22] is unaffected and may well be 0(n). Conversely, a polygon which is 2-link 
convex (such that any two of its points are at link distance 2 or less) may fail to be 
/c-convex for sublinear k. The star-shaped polygon in Figure |3] (left) is an example. 

There is, interestingly, a relation to k-point convexity as defined in [32]. A polygon 
P is called /c-point convex if for any k points pi, . . . ,pfc in P, at least one of the closed 
segments piPj belongs to P. Every /c-point convex polygon P is (A; — l)-convex. To verify 
this, we prove that if P is not {k — l)-convex, then P is not /c-point convex: Because P 
is not [k — l)-convex, there exists a line i which intersects P in at least k components. 
If we select a point in each component, it is clear that none of the segments defined by 
them is inside P, and therefore P is not fc-point convex. However, no implication exists 
in the other direction. For example, the 2-convex polygon in Figure [5] fails to be /c-point 
convex ioi k < |. Also, any /c-point convex polygon can be expressed as the union of 
m convex polygons, where m depends (exponentially) on k but is independent of the 
polygon size n; see [B]. Such a property is not shared by fc-convex polygons, as can be 
seen from the 2-convex polygon in Figure O 




Figure 5: Every white dot requires a different guard 

The class of /c-convex polygons also differs from the class of k-guardable polygons 
defined in [1]. It is known that any simple polygon with n vertices can be guarded with 
at most [|J guards [25| [3T]. The example in Figure [5] shows that this number of guards 
can be already necessary for 2-convex polygons. This is one of the reasons why most 



tools developed to study guarding problems of polygons are not very useful in the study 
of modem illumination problems [1]. 

Pseudo-triangles are polygons with exactly three convex vertices, joined by three 
reflex side chains. Any pseudo-triangle is 2-convex: If a straight line crosses a side chain 
twice, then it can cross each of the remaining two side chains at most once (Figure El 
left). That is, the stabbing number of a pseudo-triangle is four or less. In the same way 
as a triangulation defines a partition of the underlying domain into convex polygons, any 
pseudo-triangulation [28] or any pseudo-convex decomposition |3] gives a partition into 
2-convex polygons. It is an open problem (see Problem(e) in the final section) whether 
it is possible to subdivide a polygon with n vertices into a sublinear number of 2-convex 
polygons. If Steiner points are disallowed, then a 2-convex partition may have to consist 
of G(n) parts; see Figure |6] (right). 





Figure 6: Pseudo-triangle and 2-convex partition 

Two other natural questions for /c-convex n-gons are decomposing them into few 
convex pieces in terms of k, as well as giving a bound on the number of pieces of the 
convex hull minus the polygon. However, in both cases the answer is unrelated to k, 
because the polygon in Figure [7] (left) is 2-convex yet requires n — 2 convex pieces, which 
is obviously tight because every polygon can be triangulated. On the other hand, the 
polygon in Figure [7] (right) is also 2-convex and has ^ pockets, which is tight because 
two consecutive points on the convex hull are at least two edges apart if they are not 
adjacent on the polygon boundary. 

2.2 Recognition complexity 

Let us start our considerations by observing that the stabbing number of a polygon with 
n vertices can be easily found in O(n^) time, as follows. The standard duality transform 
maps each edge of the polygon to a double wedge consisting of lines through a common 
point, not including the vertical; the two lines that bound this dual wedge correspond to 
the endpoints of the primal segment. In the primal, any point inside a wedge is a line 
that stabs the segment. Therefore, the primal line that would stab most segments is in 
the dual a point that belongs to as many double wedges as possible — a maximum depth 
problem that can easily be solved by constructing the arrangement and then traversing 
its cells. 

The obtained O(n^) time bound is essentially tight, as in this section we prove that 
finding the stabbing number of a polygon, or, equivalently, finding the smallest k for 
which the polygon is /c-convex, is a 3SUM-hard problem. This family of problems is 
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Figure 7: A polygon with many convex pieces (left) and many pockets (right). 

widely believed to have an Q{n^) lower bound for the worst case runtime [l71[T9]. We 
start by giving the following result, which follows directly from Theorem 4.1 in [T7] . 

Lemma 2. For every integer a, let us consider the point pa = (a, a^) on the cubic y = x^ . 
Then if a, b, c are distinct integers, Pa,PbiPc ore collinear if and only ifa + b + c = 0. 

Proof. The points pa,Pb,Pc are collinear if and only if the determinant 

,3 1 



a a 
b b^ 
c c^ 



{b — a){c — a){c — b){a + b + c) 



vanishes, which, the numbers being different, happens exactly when a + 6 + c = 0. D 

We next show that the points Px with x G Z on the cubic y = x^ can be replaced by 
infinitesimally small vertical segments S^ with upper endpoint Px, such that three of them 
can be stabbed by a single line if and only if their three upper endpoints are collinear. 

Lemma 3. Let m, a, b, c, M be five integers such that m<a<b<c< M. Let e = 
Q(M-m) ^'^'^ ^^^ -^i ^^ ^^^ (vertical) segment with endpoints pt = (t, t^) and p[ = (t, t^ — e). 
Then Sa, Sb and Sc can be stabbed by a single line if and only ifpa, Pb, andpc are collinear. 

Proof. Assume that the points Pa,Pb,Pc are not collinear, and let us take three points 
qa = (a, a^ — £i), Qb = (b, b^ — £2), Qc = (c, c^ — £3), with < 61,62,63 < 6, i.e., three points 
on the segments Sa, Sb and Sc- The points qa, qb, qc would be collinear if and only if the 
determinant 



a a 
b b^ 



£1 
£2 



c'-es 1 



a a 
b b^ 
c c^ 



a 
b 
c 



£1 


1 


£2 


1 


£3 


1 



(6 — a){c — a){c — b){a + b + c) + 61 {b — c) — 62 (a — c) + 61 (a — b) 



is 0, but this is impossible because by Lemma 121 ;z is an integer different from 0, which 
cannot become by the addition of 5, because 



\ei{b — c) — e2{a — c) + ei{a — b)\ < \Ei{b — c)\ + \e2{a — c)\ + \ei{a — b)\ < e ■ 3{M — m) 



2' 

n 



Theorem 1. The problem of finding the stabbing number of a polygon is 3SUM-hard. 

Proof. The 3SUM problem is defined as follows: Given a set S oi n integers, do there 
exist three elements a,b,c & S such that a + b + c = 0? We will prove below that this 
problem can be reduced in O(nlogn) time to the problem of computing the stabbing 
number of an n-gon. In other words, using the notation in [17] , 



Let xi. 



3SUM ^?Co(n log n) stabbing number of a polygon. 
Xn be the input integers. We proceed to the reduction by steps. 




Figure 8: Polygon Pi with vertices on the cubic y = x^ (dashed). The scale of the axis is 
not 1:1, to make the figure visible. 

Step 1. Sort the input numbers; let yi < 1/2 < ■ ■ ■ < 2/n be the resulting list, £. This step 
is done in 0(?7,logn) time. 

Step 2. If appears thrice in C, exit with a sum of three numbers in the input being 0, 
otherwise continue. The step is completed in linear time. 

Step 3. If a 7^ appears at least twice in C, check whether —2a G C. If so, exit with a 
sum of three numbers in the input being 0, otherwise continue. The step is completed 
in 0(?T,log?T,) time, as binary search can be used in the sorted list (in fact, 0{n) time is 
sufficient by scanning the list from left to right and from right to left, in a coordinated 
simultaneous advance). 

Step 4. Remove multiples from the list C so that each number appears exactly once. This 
requires linear time. Let Oi < 02 < ■ ■ ■ < «< (where t < n) be these numbers. 

Step 5. Define m = ai — 1, M = a* + 1, and q = (M, m^). Now let us consider the polygon 
Pi whose vertices, described clockwise, are PmPaiPa2 ■ ■ -PatPMQ, where p^ = {x,x^), as in 



Pa = {a,a^) ^ ^ 



Ua = {a - e'^ , {a - s'^f) 





{aja"^ ~ e) 



Figure 9: Polygon P2 with vertices on the cubic y = x^ replaced by infinitesimal vertical 
slots. 

the preceding lemma (Figure [H]). Observe that the stabbing number of Pi is 4, and that 
the polygon can be constructed in 0{n) time. 

Step 6. Next we modify the polygon Pi to become the polygon P2 whose vertices, de- 
scribed clockwise, are PmUa^Pa^Va^Ua^Pa^Va^ . . . UatP'a.VatPMq, wherc p'^ = (x, x^ - e), M^ = 
{x — e"^, {x — e'^)^),Vx = (x + e^, (x + e^)^)); see Figure[9l This polygon can be constructed 
in 0{n) time, and in the vicinity of the point Pa^ its stabbing number changes locally from 
1 to 3; see Figure [101 Let s^. be the segment joining Ua^ to p'^.. Therefore, the stabbing 
number of P2 is 12 if and only if three of the segments s^^ can be simultaneously stabbed, 
and 8 otherwise, as two of those segments can always be stabbed. 





Figure 10: From Pi to P2 the stabbing number locally increases from 1 to 3. 

Step 7. Compute the stabbing number of polygon P2 using the best possible algorithm. 

Step 8. If the stabbing number of polygon P2 is 10, conclude that there were three 
numbers in the initial input such that their sum is 0; if the stabbing number of P2 is 8, 
conclude that there are no such three numbers. 

The correctness of Step 8 is a direct consequence of Lemma H] and Lemma O As 
all steps but Step 7 have overall complexity O(nlogn), we conclude that Step 7, the 
computation of the stabbing number of a polygon, is a 3SUM-hard problem, as claimed. 

D 

As an immediate consequence we obtain: 

Corollary 1. The problem of deciding the smallest num,her k such that a given polygon 
is k-convex is 3SUM-hard. 

Also, if we replace Step 7 in the proof of Theorem [T] by checking whether or not 
polygon P2 is 4-convex, we get: 

Corollary 2. The problem of deciding whether a given polygon is 4-convex is 3SUM-hard. 



2.3 Fast triangulation 

Triangulating a simple polygon faster than in O(nlogn) time with a simple method is a 
challenging open problem. For fc-convex polygons, this can be achieved, by the fact that 
we can sort the vertices of a /c-convex polygon P in any given direction (say, x-direction) 
in 0{kn) time: Simply scan around dP and use insertion sort, starting each time from 
the place where the x-value of the previous vertex has been inserted. Then any fixed 
value Xj, once being inserted, takes part in later comparisons at most 2k — 1 times because 
otherwise, the vertical line x = Xj would intersect P in more than k components. Having 
x-sorted P's vertices, a simplified plane sweep method can be used to build a vertical 
trapezoidation [9l[T6] (and then a triangulation) of P. Only trivial data structures may 
be used, as the scenario on the sweep hne is of complexity 0{k), by the fc-convexity of P. 
Thus, each vertex of P can be processed in 0{k) time during the sweep. We conclude: 

Proposition 1. Any k- convex polygon can be triangulated in 0{kn) time and 0{n) space. 

Using suitable data structures, a faster yet still implementable algorithm is possible, 
as we show next. Call a polygon k-monotone if every vertical line intersects the interior 
of the polygon in k intervals. (This property is implied by /c-convexity, and is equivalent 
to x-monotonicity for k = 1.) Actually, we do not even need the polygon to be simple, we 
just need a sequence of x-coordinates such that every other x-coordinate comes between 
at most 2k consecutive pairs of x-coordinates. 

Lemma 4. The vertices of any k-monotone polygon can be x-sorted in 0(?ilog(2 + k)) 
time. 

Proof. We use a binary insertion sort, in which we add the points in order along the 
polygon into a balanced binary search tree. The binary search tree has the dynamic 
finger property: inserting an element that has rank r different from the previously inserted 
element costs only 0(log(2 + r)) time. (For example, splay trees [30] and Brown & Tarjan 
finger trees [7] both have this property.) Once the elements are inserted into the binary 
search tree, we simply perform a linear-time in-order traversal to extract them in sorted 
order. 

Now we find the bound on the total cost of the insertions. When we insert an element 
of rank difference r from the previously inserted element, we can charge this cost to r 
points formed from projecting all vertices onto all edges of the polygon. There are at 
most 0{nk) such points of projection, so the total number of charges is at most 0{nk). 
Thus the total insertion cost is 0(^"^^log(2 -|- r^)) where ^"^i^-j = 0(nk). Such a 
sum is maximized when the r^ are all roughly equal, which means that they are all 
Q{{nk)/n) = Q{k). Therefore the total cost is at most 0(nlog(2 + k)). D 

To see that this bound is optimal in the comparison model, consider the case in which 
the polygon is a comb with k tines. Then sorting the x-coordinates is equivalent to 
merging k sorted sequences of length n/k, which is known to take 0(?7,log A;) time in the 
worst case. 

Lemma H] yields a fast triangulation method for general fc-convex polygons. We first 
sort the vertices of the /c-convex polygon P in a fixed direction, in 0{n log k) time. Again, 
a plane sweep is used to compute a triangulation of P. As the intersection of P with the 

10 



sweep line is of complexity 0{k) only, by the /c-convexity of P, each of the n vertices of P 
can be processed in 0(logk) time during the sweep. 

Theorem 2. Any k-convex polygon can be triangulated in 0{n\ogk) time and 0{n) 
space. 

3 Two- Convex Polygons 

3.1 Characterization 

In this section we give a characterization of 2-convex polygons that allows their recognition 
in time 0(nlog?T,), and a description of their structure that will be used later in several 
of our results. 

We observe that /c-convexity is a property that may be lost by small perturbations on 
the positions of the vertices of a polygon. For example, small changes in the positions of 
the vertices pi, . . . , p4 in Figure IH could yield a 1- or 2-convex polygon. As a consequence, 
stabbing a polygon along its edges will not, in most cases, give enough information for 
deciding its /c-convexity. 



cg. 




Figure 11: Constructing an inner tangent 

Let P be a simple polygon, and denote its boundary by dP. A line L is called a 
j-stabber of P if L crosses dP at least j times. Note that a j-stabber may contain whole 
edges of P; these are not considered to contribute to the count. An edge of a polygon P 
is called an inflection edge if it joins a convex and a reflex vertex of P. An inner tangent 
of P is a line segment T G P that contains two non-adjacent reflex vertices of P in its 
relative interior; see Figure fTTl 

Lemma 5. A simple polygon P is 2-convex if and only if P has no inner tangent, and 
no 3-stabber that contains an inflection edge. 

Proof. Consider the 'only if implication flrst. Suppose that for some inflection edge e 
of P, its supporting line L is a 3-stabber for P. Then a slight perturbation of L gives 
rise to a line L' that crosses dP at least two more times than L does. One of these extra 
crossings lies in the interior of e, and one on the second edge of P incident to the reflex 
vertex of e. Thus P admits a 6-stabber, and therefore P cannot be 2-convex. Similarly, 
if P has some inner tangent T, deflned by reflex vertices u and v, say, then a 6-stabber 
cutting off u and v exists in the vicinity of T. 
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Figure 12: Constructing a 3-stabber 

To prove the 'if implication, assume that P is not 2-convex. Then there exists a 
6-stabber L of P. Let Ci, . . . , Cg denote six consecutive crossings of L with dP such that 
the hne segment joining ci to C2 is contained in P. Two types of crossing pattern arise, as 
shown in Figures fTTl and fT2| respectively. In the former case, we simply connect the two 
crossings Ci and Cg with a geodesic path inside P (dashed), and obtain an inner tangent 
(bold). In the latter case, if an inner tangent exists locally, then we can construct it 
in a similar way, possibly by moving the geodesic's endpoints from ci and cg closer to 
each other along dP. Otherwise (as Figure [12] illustrates) we can find an inflection edge 
(bold) between ci and C3 whose supporting line L' crosses P's boundary twice between C4 
and C5. But the line L' has to cross dP once more, and thus it is a 3-stabber containing 
an inflection edge, which is not possible. D 

3.2 Recognition 

Suppose that we want to decide if a polygon P is 2-convex. (Assume that P is not convex; 
the problem is trivial, otherwise.) Our recognition algorithm is based on Lemma O We 
look for inner tangents and 3-stabbers at inflection edges. To this end, we first shoot two 
rays at each reflex vertex u of P in the directions determined by the edges of P incident 
with V. This can be done in 0(n log n) [8]. If dP is intersected more than once by any of 
these rays, then a 6-stabber exists, and we report that P is not 2-convex. 

Suppose then that each ray shot at the reflex vertices of P yields a unique intersection 
point with dP. We store the points of intersection and use them to check for inner 
tangents. Define, for each reflex vertex v of P, its critical range C{v) as the set of all 
points X G dP such that xv can be prolonged to a line segment tangent to P at v. Note 
that such a segment need not lie entirely in P. However, C{v) consists of exactly two 
connected intervals on dP whose endpoints are among the stored points obtained from 
ray shooting. See Figure [T^ where C{v) is drawn with bold lines. 

Lemma 6. P admits an inner tangent if and only if P has two reflex vertices v and v' 
such that V G C{v') and v' G C{v). 

Proof. Clearly, if two vertices v and v' define an inner tangent for P, then v G C{v') and 
v' G C{v) hold. Conversely, assume that both inclusion conditions are fulfilled. Consider 
the geodesic path tx between v and v' inside P. Either vr is the line segment ww', and thus 
can be extended to an inner tangent at v and i;', or vr detours via reflex vertices Wi, . . . ,w 



m- 
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Figure 13: Critical range for vertex v 

In the latter case (which is illustrated in Figure [T^ , we can extend the line segments vwl 
and WmV' to inner tangents, because ray shooting at reflex vertices of P has led to unique 
intersection points with dP, and thus the path vr does not cross the four (dashed) rays 
depicted in the figure. D 




Figure 14: Geodesic path between v and v' 

The strategy for detecting inner tangents is now clear. First we associate to each reflex 
vertex v two intervals, each containing a set of consecutive vertices of P lying within the 
critical range of v. We then choose a point x in dP and calculate the set R{x) of reflex 
vertices containing x in their critical ranges. We then slide x along dP, maintaining the 
set R{x) in a range search tree. R{x) has to be updated each time x passes over a point 
of intersection of dP with a ray shot from a reflex vertex of P. Moreover, when x reaches 
some reflex vertex v of P, we check in logarithmic time whether the intervals previously 
associated to v contain any element in R{v). Thus this phase takes 0(?T.logn) time. 

Since range search trees can be implemented in linear space, we have the following 
result: 

Theorem 3. Deciding if a simple polygon P with n vertices is 2-convex can be done in 
0{n\ogn) time and 0{n) space. 
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3.3 Shape structure 



We have given a geometric characterization of 2-convex polygons, in Subsection 13.11 The 
present subsection aims at giving a quahtative description of their shape. 

Lemma 7. Let P be a 2-convex polygon. Let C = poPi ■ ■ -Pt be the chain of vertices that 
connects (counterclockwise) two consecutive vertices po,pt on the convex hull CH{P). 
Then C can be partitioned into three chains Ci = poPi . . .pr, C*2 = Pr+i ■ ■ -Ps, o,nd C3 = 
Ps+i ■ ■ -Pt, for < r < s < t, such that all vertices in Ci and C3 are convex (in P), while 
all vertices in C2 are reflex. 

Proof. If C2 is empty, the lemma is obviously true, so we assume that the chain C contains 
at least one reflex vertex. Let pk be the last point of the chain C to be hit in a parallel 
sweep starting with the line defined by p^pt which, without loss of generality, we assume 
to be horizontal. Observe that pk is necessarily a reflex vertex. Let us recall that an 
inflection edge is adjacent to a reflex vertex and to a convex vertex and that, according 
to Lemma O 2-convex polygons do not have 3-stabbers containing an inflection edge. We 
are going to see that there is at most one inflection edge in the chain pq . . .p^, and that 
the same applies to the chain p^ ■ ■ -Pt- Let PiPi^i be the first inflection edge starting from 
Pq (see Figure fT5l a). As the directed ray PiPi+i intersects the chain ptpo at least once, the 
chain Pi+iPt is necessarily to the right of the oriented line PiPi+i. Observe that pt is to 
the right of the oriented line piPi^i, because otherwise a rotation around pi starting from 
Pi-iPi will define an inner tangent, forbidden in 2-convex polygons according to LemmaO 
Finally, if we assume that there exists a convex vertex pj in the chain Pi+iPk, then we 
have that Pj-iPj is an inflection edge supporting a 3-stabber: the ray PjPj-i intersects 
the polygon at least once, while the ray Pj^iPj intersects the chain p^pt and therefore 
intersects the polygon at least twice. D 

Observe that the chains Ci and C3 might be singletons in some cases, and that C2 
might be empty. However, the generic aspect of a pocket and the shape of a 2-convex 
polygon are as shown in Figure [T^b). The next result follows directly: 

Corollary 3. If the convex hull of a 2-convex polygon P has k vertices, then the boundary 
of P can be decomposed into 2k convex chains. 

The Erdos-Szekeres Theorem says that every set of n points in general position con- 
tains at least log n points that are in convex position, and that this value is asymptotically 
tight [12]. As every point set can be 'polygonized', one cannot expect a better value when 
the points are chosen from the set of vertices of an arbitrary polygon. However, when 
a point set is the set of vertices of a 2-convex polygon, we can improve this bound as 
follows. 



Theorem 4. Every 2-convex polygon with n vertices has a subset of \^/n/2~\ vertices in 
convex position. This bound is tight. 

Proof. By Corollary |3l the boundary of a 2-convex polygon with k vertices on its convex 
hull can be decomposed into at most 2k convex chains. If A; > \^/n/2~\^ we are done, 
otherwise one of the 2k convex chains necessarily has size at least \^n/2~\ . The amoeba- 
like example in Figure ITSlb). with k = \^yn/2~\ vertices in the convex hull and 2k convex 
chains of equal size shows that this bound is tight. D 
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Figure 15: Illustration for the proof of Lemma [71 

We conclude this section with a consequence of the preceding theorem. 

Corollary 4. // an n-gon is 2-convex, then its vertices can be grouped into at most 2\^2n 
subsets, each in convex position. 

Proof. Let S{n) be the number of convex subsets needed to partition the vertex set of a 
2-convex polygon with n vertices. We show that S{n) < a^fn by induction over n. The 
induction base for n = 3 is obvious, and valid for any a > 1. By Theorem 4 we find one 
convex subset of size at least [yn/2] . Moreover, by Lemma[Tl the remaining points define 

also a 2-convex polygons and we have S{n) < 1 + S{[n — ^Jnjl] ) < 1 + aW [n — yn/2\ , 
where the last inequality comes from the induction hypothesis. To prove the lemma it 



is sufficient to show that 1 + a\J n — \/n/2 < a^fn. Standard manipulation shows that 

this is true for any a. > 2^2 and any n > 1. 

D 

4 General /c-Convex Domains 

The union or intersection of simple polygons may not be a polygon. In view of this fact, 
the issue of how the degree of convexity behaves with respect to these operations is not 
meaningful for this class of objects. In this section, we consider larger classes of sets in M^ 
for which these natural questions may be discussed. We first study some properties of 
general (not necessary polygonal) subsets of M^. 

Lemma 8. The union of a k-convex set Qi and an m-convex set Qi is a [k + m)-convex 
set, which is tight. 

Proof. The number of intersections of a line with the boundary of Q1UQ2 can be at most 
2k + 2m. On the other hand, if Qi and Q2 are disjoint and the line that gives k and m 
connected components, respectively, is the same, the value is achieved. D 
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a) 



b) 



Figure 16: Intersecting /c-convex sets 

Lemma 9. The intersection of a k-convex set Qi and an m-convex set Q2 is a {k+m — 1)- 
convex set, which is tight. 

Proof. An oriented line will cut the boundary of Qi at most 2k times and the boundary 
of Q2 at most 2m times. However, the first intersection point a does not contribute to 
the total number of cuts with Qi fl Q2 unless a G Qi H Q2, in which case it contributes 
only once instead of twice as intersection. The same happens with the last point, which 
gives the upper bound. An example proving the tightness appears in Figure HHJ^a). D 

Corollary 5. The intersection of a family of m k-convex sets is a {m{k — 1) + l)-convex 
set, which is tight. 

Proof. The upper bound follows from the preceding theorem, and a construction giving 
its tightness is shown in Figure [T6t^b). D 

Theorem 5. There is no Helly-type theorem for k-convex sets. 

Proof. We are constructing a family of m 2-convex sets such that any subfamily has 
nonempty intersection yet there is no point common to all of them. Let Qm be a regular 
polygon with m edges ei, . . . ,6^ (refer to Figure [17]). Let P* be the polygonal chain 
obtained from the boundary of Qm by removing edge Ci and an infinitesimal portion of 
ej_i and Cj+i. Finally, let us give some slight thickness to the chain so it becomes a 
polygon Pi. Notice that the polygons Pi, ... , P^ are 2-convex, the intersection HI^i ^i i^ 
clearly empty, while the intersection of every proper subfamily F is nonempty because it 
contains the intervals Cj for all those Pj ^ F. D 

The preceding lemmas apply to fc-convex sets in general, not only polygonal domains. 
(A polygonal domain is a set obtained by a finite number of union operations of simple 
polygons.) However, a significant difference appears in our next results, that are possibly 
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Figure 17: No Helly-type theorem for fc-convex sets 

the most natural to explore, because they involve transversal lines, which are precisely 
the main concept underlying the definition of fc- convexity. 

Let us recall [SB] that given a family of sets Qi, . . . ,Qm, a line i is said to be a 
transversal of the family if i has a nonempty intersection with each of the sets. When the 
sets are convex, the ordering in which they are traversed (disregarding the orientation of 
the line) is called a geometric permutation, a topic that has received significant attention 
[36] • In particular, it has been proved that m compact disjoint convex sets admit at most 
2m — 2 geometric permutations, which is tight |1U] . 

Let us consider now transversals of compact 2-convex sets. Notice that every object 
will appear at least once, but may appear twice on the transversal, which we consider as 
combinatorially different cases of the associated generalized geometric permutation. What 
is the maximum number of generalized geometric permutations a family of m 2-convex 
sets may have? 

Theorem 6. The number of generalized geometric permutations of a set of m 2-convex 
objects may be exponential in m. 



Proof. We first show that a 2-convex objects can be arbitrarily complex. A nose of an 
object O is a zig-zag sequence of a refiex and a convex vertex of the boundary of O as 
depicted in Figure ITST a). Locally a line 'normal' to the nose intersects O in two connected 
components. The shaded area in Figure [T8l( a) indicates the region which is not intersected 
by a line tangent to one of the vertices of the nose. Thus we can iteratively construct 
further noses in this region without destroying the 2-convexity of O. Figure ITST b) shows 
an example where the principle shape of O is part of a disk. Observe that when the 
radius of the disk is large enough we can arrange an arbitrary number of fiat noses such 
that O stays 2-convex. 

Let Ri be an object which has the base shape of an axis-aligned rectangle, where the 
left side is actually part of a circle with sufficiently large radius and a center point far to 
the right of Ri. We place 2*~^ noses along this side, so that Ri stays 2-convex as described 
above. Next we arrange k objects Ri to Rk from left to right, as depicted in Figure ITSTc) 
for k = 3. We position the noses for each Ri in a regular way such that a rotating line 
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Figure 18: (a) and (b): 2-convex objects can have an unbounded number of noses, (c): 
The number of generahzed geometric permutations for a set of 2-convex objects can be 
exponentiaL 




Figure 19: The number of generahzed geometric permutations for a set of 2-convex 
polygons can be quadratic. 

(see the dashed hues in Figure [THT c)) intersects the noses in the same manner as the 
digit "1" shows up in the sequence of all 2^ binary numbers of length k. Thus, we get 
2^ different generalized geometric permutations for this setting, as each object appears 
twice if the nose is intersected, but only once otherwise. D 

Theorem 7. The maximum num,her of generalized geom,etric perm,utations of a set of 
2-convex polygonal domains with a total of n boundary edges is Q{v?). 

Proof. As the standard duality transform maps each edge to a double wedge, the induced 
arrangement of 2n lines in the dual plane yields a quadratic number of cells that bound 
from above the number of possible ways of stabbing the set of objects. 

To see that the bound is asymptotically tight, we give a construction using n 2- 
convex polygons; in fact, the simplest possible ones, namely, nonconvex quadrilaterals. 
Let Q\ = albiCidi be the quadrilateral shown in Figure [191 Let ^ be a horizontal line 
through al and let Q* = a*biCidi be translates of Qi in such a way that all of them 
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are pairwise disjoint and aj, Og, . . . , a* appear in this order on h. Finally, let us perturb 
infinitesimally a* to Oj in such a way that 

a) points ai, 02, . . . , a^ are in general position, 

b) for all i,j, k, with i 7^ j, the line ajCj leaves above the point c^ and below the point 

For i = 1, ... ,n, let Qi be the quadrilateral with vertices aibiCidi. There are (2) lines of 
the type ajOj; each of them leaves above and below a different set of points a^, and is a 
transversal because it crosses all the segments hkCk for every k. Now, if a^ is below the 
transversal, Qk is intersected once, while if a^ is above the transversal, Qk is intersected 
twice. Therefore, we have obtained (2) generalized geometric permutations. D 

Observe that the two preceding theorems apply mutatis mutandis to /c-convex sets, 
because 2-convex sets are also /c-convex for k> ?>. 



5 Discussion and Open Problems 

In this paper we have considered a new concept of generalized convexity. Moving from 
convexity to 2-convexity is seemingly a small change, as we are just accepting lines to 
intersect in at most two connected components instead of one. It is remarkable that this 
modest departure has strong consequences in the complexity of the new class of objects, 
as we have seen in this paper; obviously, even more when the degree of convexity is 
increased. Several open problems remain and many interesting questions can be raised. 
We list some of them below. 

a) Can the recognition of 2-convex polygons be carried out in linear time, improving 
on the 0(nlog?T,) algorithm we provide? 

b) Finding the smallest k such that a given polygon is /c-convex is a 3SUM-hard prob- 
lem. In particular, recognizing 4-convexity is already 3SUM-hard. We do not know 
whether the situation is the same for 3-convexity or whether a subquadratic time 
algorithm exists for this case. 

c) Is it possible to generalize Theorem HP For example, is it true that every fc-convex 
polygon with n vertices has a large subset of vertices that are the vertices of a 
{k — l)-convex polygon? 

d) Let us define the /c-convex hull of a point set S as the smallest area polygon which 
is /c-convex, has a subset T d S as vertex set and every point in S* \ T is inside the 
polygon. Which is the complexity of computing this /c-convex hull? Observe that 
for k = 1 this notion is the usual convex hull of a point set. 

e) Give combinatorial bounds and efficient algorithms for decomposing a polygon into 
/c-convex subpolygons. This is a classical problem when convex subpolygons are 
considered [18] and also the decomposition into pseudotriangles, which are 2-convex 
polygons, has been studied [28]. However, the latter result might be improved by 
considering more general 2-convex polygons. 
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f) A fc-convex decomposition of a set S oi n points in the plane is a decomposition 
of its convex hull into /c-convex polygons such that every point in S* is a vertex of 
some of the polygons. For k = 1, a. triangulation suffices, though it has been proved 
that if we allow arbitrary convex sets the number can be reduced [23]. On the 
other hand, it has been shown that there exist always a decomposition into exactly 
n — 2 pseudotriangles, which are 2-convex polygons [28]. It is an intringuing open 
problem to decide whether this number can be reduced to sublinear if we allow 
arbitrary 2-convex polygons. 

Finally, let us mention that in this paper we have focused on /c-convex polygons. It is 
natural to define a similar concept for finite point sets, namely, being in k- convex position, 
where k is given by the smallest degree of convexity attained when all possible polygoniza- 
tions of the point set are considered. This issue is considered in a companion paper [2] . 

Acknowledgements We would like to thank Thomas Hackl, Clemens Huemer, David 
Rappaport, Vera Sacristan and Birgit Vogtenhuber for sharing discussions on this topic. 
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